While electronic documents stored on computers provide a number of advantages over written documents, many users continue to perform some tasks with printed versions of electronic documents. These tasks include, for example, reading and annotating the documents. With annotations, the paper version of the document assumes particular significance, because the annotations typically are written directly onto the printed document. One of the problems, however, with directly annotating a printed version of a document is the difficulty in later converting the annotations into electronic form. Ideally, electronically stored annotations should correspond with the electronic version of the document in the same way that the handwritten annotations correspond with the printed version of the document.
Storing handwritten annotations in electronic form typically requires a user to review each handwritten annotation and personally enter it into a computer. In some cases, a user may scan the annotations written on a printed document, but this technique creates a new electronic document. The user must then reconcile the original version of the electronic document with the version having the scanned annotations. Further, scanned images frequently cannot be edited. Thus, there may be no way to separate the annotations from the underlying text of the original document. This makes using the annotations difficult.
To address this problem, pens have been developed to capture annotations written onto printed documents. In addition to a marking instrument, this type of pen includes a camera. The camera captures images of the printed document as a user writes annotations with the marking instrument. In order to associate the images with the original electronic document, however, the position of the images relative to the document must be determined. Accordingly, this type of pen often is employed with specialized media having a pattern printed on the writing surface. The pattern represents a code that is generated such that, the different section of the pattern that occur around a location on the media will uniquely identify that location. By analyzing or “decoding” this pattern, a computer receiving an image from the camera can thus determine what portions of the code (and thus what portion of a document printed on the paper) were captured in the image.
While the use of such patterned paper or other media allows written annotations on a paper document to be converted into electronic form and properly associated with the electronic version of the document, this technique presents its own difficulties. For example, the printed document itself may obscure areas of the pattern printed on the writing surface of the media. If the pen captures an image of one of these areas, then the computer may not be able to use the pattern to accurately determine the location of the document portion captured by the image. Also, the computer may not accurately recognize the code from the image. For example, if the code is binary, then the computer may erroneously recognize a portion of the pattern representing a “0” value as a “1” value, or vice versa.
Further, in some situations the unique positioning properties of the code cannot be utilized because the code values detected from the pattern are not consecutive, or the values do not have sufficient bits to uniquely identify a section of the code. Moreover, in order to stay in synchronism with the movement of the pen, the pattern captured in an image must be decoded within fixed time period. For example, if the pen captures about 100 images per second, the decoding time for each frame cannot exceed 10 ms. The possible solution candidates must therefore be generated and analyzed at a fast rate.